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The scope of institutional research (IR) undertaken in Australasian universities is progressively 
expanding. A traditional focus on student life cycle elements such as enrolment, retention and 
satisfaction has been complemented for some years now by other areas of focus including research 
performance and community engagement. More recently, university rankings and national policy- 
driven topics such as academic standards have joined the agenda. In Australia, performance-based 
funding for student outcomes has been introduced, to be joined by deregulation of the student market 
from 2012. In New Zealand, the reform agenda is more progressed in some respects, with the 
introduction of student demand-driven funding in the 1990s. 

Reasons for this broadening IR agenda in both countries include increased competitive pressures 
within the higher education sector, increased external scrutiny of institutions by regulators, funding 
bodies and rating agencies, and the development of more comprehensive performance indicator 
frameworks, both within institutions and by governments. At first pass, this menu of research topics 
and drivers might suggest that IR in Australasia has indeed ‘come of age’. While this is clearly the 
case in terms of relevance, it is contestable whether this is also the case in terms of maturity and 
capacity: it is in this regard that practitioners risk ‘coming unstuck’ (a reference to the theme of the 
AAIR 2010 Forum). This article explores these issues drawing on the experiences of two Australasian 
universities, one from each side of the Tasman Sea. 

The authors suggest that three types of factors—structural, process and cultural—influence the ability 
of institutions to maintain a sustainable and effective IR function and to undertake proactive analysis. 
They conclude that there is a range of tools now available to practitioners and leaders to allow their 
institutions to adjust to the new realities of the higher education sector. These tools point to a 
sustainable operating model for the future, focused on supporting strategic agenda-setting rather than 
conducting historical analysis. 


This paper was first presented at the Annual Conference of the Australasian Association for Institutional 
Research, Has institutional research come of age in Australia? in Geelong, 10-12 November 2010. 
Correspondence to: Martin Hanlon, Director, Planning and Quality, University of 
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Defining the Current Challenges 

AAIR defines the purpose of IR as being ‘to provide objective , systematic and 
thorough research that supports the institution’s enrolment goals, planning, policy formation 
and decision-making’ (emphasis added) (AAIR, 2010). 

This purpose statement provides three qualifiers to distinguish quality research from 
substandard or superficial research: ‘objective’, ‘systematic’ and ‘thorough’. It also scopes the 
topic areas to which this discipline can be applied. While the first area relates to a specific 
function (‘student enrolment goals’), the other three (‘planning, policy formation and 
decision-making’) are generic and can be applied to all of an institution’s endeavours, both 
strategic and operational. The conclusion is that IR should aspire to be both deep in rigour and 
broad in application. 

Turning to the activities of IR, these include ‘the collection, analysis and 
interpretation of information descriptive of an institution and its activities, including its 
students and staff, programs, management and operations’ (emphasis added) (Zimmer, 1995). 
For practitioners, these definitional statements present a number of challenges to the delivery 
of IR that can effectively achieve its ultimate purpose, that of informing decision-making. 
Matulick (2007) suggests that decision-making in this context is focused on maximising 
institutional performance, as monitored by tools such as key performance indicators (KPIs), 
and maximising funding of the institution through tools such as student load forecasting 
models and databases to capture research output and impact measures. 

The authors suggest that to be effective in the modern era, practitioners need to: 

• have the capability and the capacity to undertake rigorous research, as implied by 
‘systematic’ and ‘thorough’ 

• be knowledgeable of their institution’s planning and policy agenda, or at least 
strategic elements of it, as it affects both students and staff, and ideally other key 
stakeholders such as employers 

• have access to information necessary to allow the initial research step of collection to 
occur and the subsequent value adding steps of analysis and interpretation 

• understand and have access to the decision-making processes of the institution. 

In the context of more rapid cycle times for the generic ‘Plan, Do, Check, Act’ quality 
management sequence, a fifth challenge needs to be added to the list: the ability to regularly 
meet challenges 1-4 above in a timely manner. This additional element can establish a tension 
between rigour and timeliness, highlighting the risk of ‘overanalysis’ at the expense of 
‘decision useful’ inputs. 

The challenges described above would be seen by most institutional researchers as 
formidable in their own right in the context of a relatively static research agenda. Such an 
agenda could be characterised by stability in the competitive landscape, government policy 
settings, funding streams and institutional leadership and direction. In such a context research 
demands might be largely cyclic and predictable. However, in recent years there would be 
few institutional researchers who would report that all or even any of these characteristics 
were applicable to their institutions. Arguably the most significant contributor to a dynamic 
research agenda is government policy. 
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Policy Contexts in Australia and New Zealand 

As noted above, the last decade has seen a steady expansion in the IR agenda, 
particularly for larger institutions with multidimensional missions. The growing complexity 
of this agenda is being driven by continuously evolving national policy in Australia and New 
Zealand. A brief comparison of policy settings may help to situate the challenges facing 
Australasian institutional researchers. 

Higher education policy in Australia in the 2000s saw the Commonwealth introduce 
and evolve performance-based funding schemes for research and research training (e.g. block 
funds for general research and research training infrastructure) and coursework programs (e.g. 
Learning and Teaching Performance Fund). These schemes further solidified the broader 
move towards a performance-driven funding model for the Australian higher education sector 
(Wells, 1996). 

Australia’s federal Labor government came to power in 2007 committed to delivering 
an ‘education revolution’. Following the Bradley and Cutler reviews completed in 2008, the 
government translated this vision into specific goals, such as participation targets for all 25- 
34-year-olds. It coupled these goals with a plan to deregulate undergraduate government- 
supported student load, leading to funding on the basis of student demand from 2012. The 
government also commissioned the ‘Excellence in Research for Australia’ (ERA) process, 
with a strong emphasis on the quality of research undertaken across the sector. The 
introduction of mission-based compacts in 2011 to foster greater diversity in the sector and 
the scheduled launch of the My University website in the near future will consolidate the 
accountability dimensions to this reform agenda. Also significant will be the progressive 
establishment of the Tertiary Education Quality and Standards Agency (TEQSA) from mid- 
2011, a new independent national regulatory and quality agency for higher education, and its 
use of a proportionate risk-based approach to regulation linked to national minimum 
standards. Finally, since the August 2010 federal election with independent members of 
parliament holding the balance of power in the House of Representatives, ‘pathways to 
university’, particularly in regional areas, has gained new prominence. 

New Zealand’s reform agenda has varied from Australia’s, with a deregulated student 
demand-driven funding framework introduced in the 1990s. Since 2003, reforms have been 
guided by the Tertiary Education Commission (TEC), which had an initial focus on 
addressing spiralling costs and the proliferation of duplicated or poor quality lower-level 
qualifications (largely in the nonuniversity tertiary sector) that resulted from the deregulated 
student load-based funding system of the 1990s. TEC’s more recent focus has shifted toward 
making the tertiary sector more responsive in its contribution to the government’s overarching 
social and economic goals, as expressed in the government’s Tertiary Education Strategy 
(TES). 


The TEC has progressively guided a shift toward the key priorities outlined in the 
New Zealand Government’s TES, placing increased emphasis on performance incentives and 
accountability. Principal mechanisms for achieving these policy aims have been the 
performance-based research funding (PBRF), introduced in 2003, and investment planning 
processes (three-year funding contracts), both of which bear similarities to mechanisms in 
current Australian tertiary education policy (mission-based compacts and ERA). However, 
over the next three years the shifts required of institutions in New Zealand’s tertiary sector— 
and agreed through the Investment Plan process—will occur within a capped funding 
environment, from which the Australian sector is set to emerge. 
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This brief comparison of policy settings indicates that areas of focus for institutional 
researchers, both in Australia and New Zealand, will continue to evolve, and that there are 
opportunities for cross-jurisdictional learning. 

Future Challenges and Policy Contexts 

A question facing all IR practitioners and their leaders seeking to bed down a 
reasonably future-proof approach to IR is what will be the policy settings and broader 
operating context over the medium- to long- term future? One insight into the potential policy 
settings of future Australasian governments is provided by Galwey and Ogilvie (2010). Table 
1 presents a desired future state in a New Zealand context (although adaptable to the 
Australian situation), described as a state that would enable institutions to readily adapt to 
changes in their environment and strengthen their likelihood of business sustainability. The 
authors of the article note that the paradigm shift described will not eventuate if it proves to 
be unpalatable to public policymakers or those charged with policy delivery. Indeed, many of 
the current policy settings in either Australasian jurisdiction do not signal a clear move 
towards this future scenario, although some elements may be emerging (e.g. self-governance 
in Australia through the mission-based compacts process). 

Table 1 

Emergent Change in the New Zealand Tertiary Education Sector 


Aspect 

Current 

Future 

Emphasis 

Risk - minimising the Crown's residual liabilities 

Optimisation - maximising the Crown's strategic 
outcomes 

Time frame 

Short/medium term 

Medium/long term (especially given capital asset 
management considerations) 

Success 

Viability 

Sustainable value creation 

Managerial emphasis on TEIs 

Control 

Self governance 

Performance management 

KPIs, support control 

Balanced Scorecards, support organisational 
learning and continuous planning 

Risk propensity 

Low 

Higher, with stronger emphasis on identification 
and mitigation 

Culture 

Low trust, rules based 

High trust, self governance 

Engagement 

At some distance 

Partnering 

Analytical focus 

Historically orientated 

Strategic and agenda setting 

Resource allocation 

Budgets central to process, fixed resourcing 

Rolling forecast central to process, dynamic 
resourcing 

Adaptability (of ITPs) 

Slow 

Shifts quickly to market and policy changes 

Sector behaviours 

Compliant, observing the rules 

Committed and aligned to outcomes 

Benchmarking 

Target setting 

Catalyst for continuous 
improvement/organisational learning 

Strategy 

Is a destination 

Is a direction 


Note: TEIs = Tertiary Education Institutions; ITPs = Institutes of Technology and 
Polytechnics 

From: Galwey D., & Ogilvie, B. (2010). An engagement framework for managing the 

crown’s ownership interests in the New Zealand tertiary education sector. Measuring 
Business Excellence , 14(1), p. 68. Copyright 2010 by Emerald Group Publishing Ltd. 
Reprinted with permission. 
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Institutional Responses: Two Australasian Case Studies 

The University of Technology, Sydney (UTS) and Auckland University of 
Technology (AUT) are two city-based technology universities of similar profile and scale. 
These institutions present useful case studies of universities with comparable goals and 
challenges but operating in different jurisdictions and at different points in the development of 
integrated business intelligence (BI) solutions. 

Both universities have globally focused purpose statements: UTS ‘To advance 
knowledge and learning to progress the professions, industry and communities of the world’ 
(UTS Strategic Plan 2009-2018), and AUT ‘To foster excellence, equity and ethics in 
learning, teaching, research and scholarship and in so doing serve our regional, national and 
international communities’ (AUT Strategic Plan 2007-2011). Both have a technological 
heritage and an applied focus, with distinctive contributions in disciplines such as 
engineering, information technology, business and creative industries. Each institution has 
articulated critical success factors in the areas of learning outcomes and professional practice, 
graduate employability, research profile, campus renewal and organisational effectiveness. 
UTS is the older and slightly larger of the two universities with 30,000 students enrolled, 
while AUT has 26,000 enrolments. AUT has been the fastest growing university in New 
Zealand in terms of student numbers and evolution of student profile from 45% pre/subdegree 
and 55% undergraduate/postgraduate students in 2000 to about 85% 
undergraduate/postgraduate students in 2010. 

UTS and AUT both have dedicated IR units: UTS has a Strategic Intelligence team 
within the Planning and Quality Unit, part of the Senior Deputy Vice-Chancellor’s portfolio, 
and AUT has an Institutional Research Unit within the Strategy and Development arm of the 
Vice-Chancellor’s Department. The UTS SI team is currently resourced with six permanent 
full-time equivalent staff for ‘business as usual’ activities while AUT has four. 

Both teams are principally responsible for the annual student enrolment forecasting 
process, enrolment scenario analytics and load planning support. They also deliver routine 
and ad hoc research and analysis to support statutory reporting, performance monitoring, 
strategic developments and decision-making processes. Added to these responsibilities is an 
annual program of university-wide student feedback and experience surveys. UTS’s survey 
program incorporates a regular survey of reputation with industry leaders, and AUT’s 
incorporates ad hoc staff feedback surveys. 

Another parallel is that both teams support their institution’s existing BI capability and 
development. The Strategic Intelligence team at UTS maintains a BI environment 
incorporating around 50 data cubes, a data warehouse and multiple dashboards supporting the 
university’s KPI framework spanning five performance domains. During 2008-2010 the team 
was supplemented with a further three full-time employees to deliver an integrated BI 
solution incorporating a data warehouse. Since 2010, AUT’s Institutional Research Unit has 
led a BI development program with a similar goal, with one additional employee assigned for 
this purpose. 

The IR model practiced in both institutions is similar: the central team works within a 
broader network. These networks include ‘IR practitioners’ in various units with particular 
areas of focus, such as strategic planning (e.g. KPI reporting and target setting), student 
services (e.g. student surveys, attrition analysis), learning and teaching support (e.g. student 
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feedback on teaching), the research office (e.g. research performance analysis) and human 
resources (e.g. staff surveys, workforce planning). 

In both institutions the volume and distributed scope of IR reflects both the 
comparatively decentralised structures as well as the broad range of research, monitoring and 
analytics that are now recognised as being essential to the effective operation of a university 
and the maintenance of quality. It is therefore encouraging that there is growing cooperation 
between these various departments, on an informal or ‘as required’ basis, reflecting the 
increasingly complex and multidisciplinary nature of some of the required analytical 
processes and outputs. 

University leaders at both UTS and AUT have recognised that the capacity of their IR 
teams to undertake a comprehensive and routine program of objective, systematic and 
thorough research and analytics is greatly enhanced if a purpose-built BI platform is 
developed. These platforms are typically designed to operate across key university data 
sources (student management, research management, human resources, finance, space 
planning) and are intended to provide access to a comprehensive, integrated and standardised 
source of data, with capability to support forecasting, predictive analytics and modelling. 

As noted earlier, a BI platform of this nature has recently been developed at UTS with 
standardised datasets supporting a wide range of university KPIs now stored in a data 
warehouse. AUT has recently allocated internal funding to support a pilot BI solution (proof 
of concept). Ultimately, both institutions foresee their BI platforms supporting exploratory 
analytics aimed at identifying opportunities to enhance student outcomes and operating 
effectiveness and efficiency. 

In the experience of both institutions, compliance requirements associated with 
reporting of student load, supporting performance-funding arrangements, preparing capital 
funding cases and responding to targeted performance reviews and quality audits consume a 
large proportion of an IR team’s capacity. Proactive internally driven analyses, such as those 
directly linked to an institution’s strategic plan, are more readily undertaken in the periods 
between compliance obligations. Mature BI platforms have the capacity to support both 
categories of analysis, with the goal of creating more time for the latter. 

Despite the reasonably coordinated approach to IR in place at both UTS and AUT, and 
the existence of senior support for further strengthening of analytical capabilities, 
practitioners at both institutions would claim that they are struggling to maintain delivery of 
high-quality services across the ever-broadening research agenda noted earlier. In both 
institutions, university leaders have expressed a desire for deeper analysis and interpretation 
across a broader range of topics, and for timely reporting. 

Institutional Responses: General Factors 

The comparisons explored here suggest that despite the different policy settings and 
historical development in Australian and New Zealand higher education, the institutional 
response—as expressed through the evolving function and operation of the case study IR 
units—has been similar. 

Looking forward, the combined impacts of an expanding institutional research agenda, 
increased attention to the outputs of IR teams, the creeping influence of BI systems in 
strategic decision-making and the continued uncertainty of government funding regimes is 
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likely to result in more considered decisions by institutions about the design and resourcing of 
their IR function. Institutions that can evaluate a range of alternatives for the design of their 
IR unit and the analytical tools appropriate to their own emerging policy-setting and then 
evaluate the efficacy of their decisions will sustain an advantage in a globally competitive 
higher education market. 

The authors suggest that three types of factors—structural, process and cultural— 
influence the extent to which an IR unit will be sustainable into the future and capable of 
undertaking value-adding analysis. 

Structural factors, both internal and external, include the scope of the institutional 
mission (e.g. teaching and learning only vs. comprehensive, public vs. private), scope and 
complexity of the student profile and fee regimes (e.g. mix of undergraduates and 
postgraduates, government-supported and full fee-paying, domestic and international), 
competitive context (e.g. geographical proximity of other providers) and demographic profile 
of the local catchment for students and staff. Additional internal structural factors include 
location of the IR team within the organisation and. of course, the team’s size and capacity. 

Process factors include the strategic planning framework (e.g. specificity of strategic 
goals) and transparency of decision points within various processes (affecting the 
opportunities for researchers to add value at the appropriate time in the appropriate way). 
Other process factors include maturity of both KPI frameworks (in terms of alignment of 
metrics to business processes and maturity of target setting) and business intelligence systems 
(in terms of data quality and timeliness, data management and control, analytical tools and IT 
capability). 

Cultural factors include the nature of the research questions being posed by 
management (e.g. ‘Are we sure the results for this indicator are reliable?’ vs. ‘Why aren’t we 
performing better than institution X on this indicator?’) and the empowerment of IR staff to 
research and present ‘frank and fearless’ objective analysis that may be critical of internal 
strategies or previous resource allocation decisions. In the words of Low (2002), leaders need 
‘to allow the facts (and the institution’s values) to interfere with some of the “good theories’”. 

These factors may provide reference points for any IR unit conducting a self- 
assessment of its operating environment as a precursor to developing an operating model for 
the future. 


Towards New Operating Models: Potential Tools for the Future 

For some time the IR community has acknowledged that practices need to evolve in 
order to adjust to the ever-changing realities of the higher education sector. A decade ago 
Palermo (2000) argued that researchers needed to become ‘information brokers’, creating 
links between people, policy, process and data, analysing the system as a whole and 
encouraging leaders to challenge their own assumptions. Low (2002) foreshadowed that IR 
practitioners would be needed for all aspects of planning, not just planning for changes in 
government-funded student load, and involve ‘teams of researchers from a number of areas 
within the university’. Kleeman and Lake (2005) reported on a case study reflecting these 
foreshadowed evolutions, where lengthy supply-driven reports were replaced by ‘just in time’ 
purpose-designed data reports feeding into agreed processes, such as reviews of courses and 
schools. Most recently Knight (2010) found that personal characteristics of effective 
institutional researchers in the current US context (as assessed by researchers themselves) 
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include engagement with various university processes, strong relationships with decision¬ 
makers, responsiveness and foundational elements such as an ability to convey information 
effectively, concern about data integrity, and technical and analytical skills. 

Galwey and Ogilvie (2010) provide a valuable contribution from the perspective of 
policymakers in New Zealand’s tertiary education sphere (readily adaptable to IR 
practitioners): an unrelenting approach to systems disciplines, including systems thinking, 
systems design and systems implementation, with an eye to anticipating unintended 
consequences of poorly designed interventions. These characteristics provide some pointers 
towards an operating model for IR practitioners in the years ahead. 

Returning to the five attributes of an effective practitioner suggested earlier (capability 
and capacity to undertake rigorous and time-consuming research; knowledge of the 
institution’s planning and policy agenda; access to information; access to the decision-making 
processes; and ability to regularly meet these four challenges in a timely manner), a 
fundamental question that arises is what tools are available to support practitioners in 
developing these attributes? 

Based on a limited literature review and the case studies presented above, the authors 
have identified a number of specific tools that IR teams and university leaders can adopt and 
combine to develop the capabilities necessary for the future. Each institution will need to 
determine which mix of tools is most likely to optimise their research capacity, taking into 
account their emerging policy context and other contextual factors. A key consideration will 
be the tensions that are likely to arise between efforts to consolidate the central IR team’s 
capability and efforts to build IR capability in other areas. This latter scenario could 
potentially dilute the capacity of the central team through increased demand for user support. 

Seven IR tools are described below, with ‘tools’ interpreted to include approaches 
relevant to both practitioners and institutional leaders. The authors suggest that these and 
other enhancements of this nature will allow institutions to more fully harness the capability 
offered by their BI solutions, and move beyond what could be considered to be the 
‘demonstration phase’. A 2008 survey of IR practitioners in the Australian higher education 
sector identified that while investment in BI solutions had increased, progress to full 
deployment stage was mixed (Australian University Quality Agency, [AUQA], 2010). 

Virtual IR Teams/BICC 

The ever-expanding scope of IR and relentless demand for timely responses to 
complex research questions require institutions to fully harness the analytical capacity of their 
workforce to address research questions. Depending on the size and scope of the institution, 
this workforce may include the central institutional researcher or research team, usually 
located in the planning, statistics and/or quality unit. Data analysts in various executive 
support roles, specialist central support areas (e.g. learning and teaching, equity, postgraduate 
research students, research, human resources) and possibly large academic units, in-house 
statisticians and external consultants may also be included. Cooperative arrangements 
between staff across this spectrum can be valuable mechanisms for spreading the research 
load. Many institutions already have such a devolved model in place. These arrangements 
tend to be informal and based on personal relationships rather than defined roles, potentially 
leading to suboptimal outcomes and duplication of effort. Formalising such arrangements in 
terms of clarifying common goals and individual responsibilities, if pursued in a cooperative 
spirit, should lead to a strengthened capacity to respond to, and anticipate, research questions 
and a more sustainable IR capacity. To be successful, such a team or Business Intelligence 
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Competency Centre (BICC) needs to be governed by commonly shared protocols for 
managing information covering topics such as data definitions and rules for changing 
datasets. 

Lead Indicators and Business Analytics 

Traditional external reporting requirements and internal performance measures used in 
the higher education sector tend to focus on output or outcome (‘lag’) indicators. They give a 
sense of how the institution performed in the recent past (or in the case of many research 
indicators, the more distant past), rather than input or process (‘lead’) measures indicating 
how the institution is likely to perform in the future. According to Davenport and Harris 
(2007), answering questions such as ‘what happened?’, ‘where exactly is the problem?’ and 
even ‘what actions are needed?’ is the domain of access and reporting. In contrast, analytics 
incorporating the use of forecasting, predictive modelling and optimisation, answers proactive 
questions such as ‘what if these trends continue?’ ‘what will happen next?’ and ‘what’s the 
best that can happen?’ Figure 1 presents one representation of the relationship between the 
range of reporting and analytical tools, and the questions they can answer. 



Figure 1 

Business intelligence and analytics capability spectrum. 

From: Davenport, T.H., & Harris, J.G. (2007). Competing on analytics: The new science of 
winning (p. 8). Boston: Harvard Business Publishers. Copyright (2007) by Harvard 
Business Publishers. Reprinted with permission. 
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Both Figure 1 and Table 1 signal that universities which chose to invest in their analytical 
capabilities through forward-looking performance measurement frameworks, advanced BI 
systems, and staff with skills in analytics, will have more opportunity to influence future 
institutional results. 

Enterprise Process Architectures and Fully Leveraged BI Solutions 

Identification of lead indicators is problematic if the processes they measure are ill- 
defined. Despite the prevalence of specific ‘process improvement’ initiatives throughout 
Australasian universities, there is little evidence of universities undertaking a holistic 
management of their processes or ‘value chains’ at the enterprise (university) level. Enterprise 
mapping to create an ‘architecture’ representing the value delivered to customers through 
‘core’ business processes is common practice in other sectors as a platform for process 
governance, measurement and improvement. An international search commissioned by UTS 
unearthed only one such documented architecture in the higher education sector, interestingly 
from another Australasian university (Hanover, 2009). As with any management tool 
developed outside of the higher education sector, the concepts and language need to be 
adapted to reflect the centrality of students and academic staff to the mission of a university, 
as distinct from classic ‘customers’, and the oft-quoted difference between operating in a 
business-like manner and as a commercial business. These limitations aside, UTS is finding 
the concept of an enterprise process architecture to be a useful analytical tool. One application 
under consideration is using the architecture to guide determination of lead indicators for core 
and support processes. The rationale is that by having a better understanding of the process 
requirements that lead to improved performance outcomes, there is a stronger likelihood of 
repeating those results. 

Student Segmentation and Identifiers 

A critical specialisation of IR in universities will always be the tracking of student 
groups of interest as they progress through the student lifecycle or ‘value chain’ from pre¬ 
enrolment to postgraduation. Such tracking allows institutions to better anticipate the needs of 
particular cohorts. The advent of data mining tools with artificial intelligence capabilities 
presents opportunities for institutions to better harness the intelligence hidden in their student 
data systems by evolving from traditional demographic cohorts to customised 
multidimensional segments. Commercial data analysis consultants now claim to offer a high 
level of capability in this area. Also of value would be the ability to track individual students 
as they move through an institution and indeed throughout a national education and training 
system. In Australia, the possible introduction of national unique student identifiers for the 
school and vocational education and training sectors presents great potential for application 
throughout the entire education system, as long as necessary data and evaluation capabilities 
are developed within institutions (Harman, 2010) and privacy considerations addressed. 

Trans-Institutional Studies 

Given the commonality of research questions across institutions, even across 
jurisdictions, there will always be a place for supra-institutional research. In fact, there is a 
growing trend to compare institutions’ performance domestically and overseas through 
common student surveys, for example (Moodie, 2009). Yorke (2010) concludes that trans- 
institutional studies can be effective at various points along the intensity spectrum from 
heavily resourced studies (macro-level) to informal online inquiries of colleagues with a 
common interest (micro-level). Studies in between these extremes (meso-level) are attractive 
in that they can provide rigour and be undertaken at relatively small cost. The overriding 
consideration of all such studies is fitness for purpose: sometimes a short, sharp, online 
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inquiry will be sufficient to inform a decision or comply with a deadline. In cases where more 
rigorous meso-level research is required, the challenge for the future is to conceive, 
commission and deliver studies in a more timely manner aligned to the collective and 
individual decision points of participants. In these circumstances, prompt agreement on a lead 
researcher, roles of any third parties, funding arrangements and protocols around data sharing 
will be prerequisites to an effective study. A number of third parties now exist with the 
capability to undertake quick turnaround research studies and more formal benchmarking 
projects. 

Enabling Decision-Makers with Self-Service Management Information 

All the tools presented above are dependent on institutional researchers initiating 
information collection, analysis and/or dissemination or development of the tool itself. 
Focusing on these dimensions alone presents a risk that the IR team becomes a bottleneck to 
the flow of relevant and timely information to decision-makers. Part of the solution to a 
sustainable IR function has to be the provision of decision-makers and their staff with self- 
service access to online information. The aim of such a service should be to allow decision¬ 
makers to answer their day-to-day questions without having to contact the IR team. This will 
allow the team to focus on answering the more complex research questions. A prerequisite for 
self-service to be well received by users is trust in the data: any sense of ‘multiple sources of 
the truth’ or that a complex set of filters need to be mastered before meaningful reports can be 
generated will undermine the acceptance of self-service. Matulick and Murdoch (2004) 
provide advice on the development of a data quality framework applicable in this context. 
Critical concepts include data governance, ownership, classification, validation (at both input 
and output) and visibility. 

While elements of self-service have been provided by several Australasian institutions 
for some time, the attributes of a successful deployment are now more fully understood. 
Functionality that continues to become easier to use, is intuitive and web-based (e.g. 
dashboards with drill down capability) will lead to better engagement of decision-makers with 
IR products and services. The end goal of any self-service strategy should be to generate 
‘more effective data consumers who leam to ask more sophisticated questions and solve more 
complex problems’ (SAS, 2010). 


Strategic Conversations 

As alluded to throughout this article, for IR to be effective it needs to go further than 
researchers simply updating their work practices and broadening their toolkits. Leaders need 
to play their part by making sure their institution’s decision-making processes are transparent, 
accessible to institutional researchers and, where possible, stable. Researchers are better able 
to focus their research efforts into areas of highest importance to the institution if they are 
exposed to how decisions are made and how IR is used (or not used) in the process. 

Many decision-making processes, both academic and administrative, would benefit 
from dialogue between decision-makers and staff with relevant institutional research 
capability. Common topics in these conversations will include the institution’s performance 
and strategic direction, internal comparisons between faculties and units, and external 
comparisons with competitors and benchmarking partners. All parties should be encouraged 
to bring ‘evidence’ to the table, and to be objective and reflective. Simple facilitation tools 
such as ‘five whys’ can assist participants to clarify the root cause of a particular outcome or, 
if unclear, the fundamental research question to be investigated. Conversations can be 
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extended to involve other institutions, either within or outside the sector, in the context of 
benchmarking for improvement. 


Conclusion 

The challenges facing IR practitioners in Australasia at the present time are 
considerable amidst an ever-expanding IR agenda, shortened cycle times and the constant of 
limited resources. However, there are a range of tools available to practitioners and leaders to 
reduce the risk of practitioners ‘coming unstuck’. These tools point to what a sustainable 
operating model for the future might look like, focused on supporting strategic agenda-setting 
and systemic approaches, rather than conducting narrow historical analyses. IR units will need 
to make choices about which tools will optimise their research capacity, and practitioners will 
need to change their conceptualisation of IR and what personal skill sets they need. However, 
for IR to truly come of age university leaders will need to work with practitioners to develop 
and resource roadmaps for development of their institution’s IR capability and capacity. They 
will also need to demonstrate that they are embracing evidence-based decision-making. 
Finally, at the national level, AAIR will have routine opportunities to promote these 
principles in Canberra and Wellington! 

Postscript: A Thought on Quality Control 

The two definitions of IR presented at the beginning of this article, if combined, 
provide the beginnings of a possible quality control checklist for IR (Table 2). This checklist 
could be applied to each discrete research task or be used more holistically. 

Table 2 

Potential Institutional Research Quality Control Checklist 


Institutional 

Research 



Institutional Research Attributes 


Objective? Systematic? Thorough? 


Was information 

0 

0 

0 

collcctionTTr'''"^ 


Was information 

0 

0 

0 

analysis . . . 




Was information 

0 

0 

0 


interpretation . . . 
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